Spatial scan replication circuit

ABSTRACT

In an image replication circuit, the improvement of replicating a given element at a certain location with the most similar of surrounding sets of image elements.

FIELD OF THE INVENTION

[0001] An apparatus and method is shown for modifying the creationand/or presentation of image information displayed, printed or createdon a raster or matrix image or graphic display or printer, therebyincreasing the apparent image quality. Means for deriving a plurality ofneighboring image elements or elements of the video signal whichneighbor in time or space to a common location, and means fordetermining the image elements replication at such location in responsethereto, are also shown. Particular embodiments relation to increasingthe apparent temporal and spatial resolution of raster scannedtelevision computer and permanent copy devices are shown.

BACKGROUND OF THE INVENTION

[0002] This invention relates to increasing the apparent temporal orspatial resolution of a created and/or displayed image which istypically produced by ordered groups of elements such as by a raster ormatrix element or device, without a required increase in the number ofimage elements of the image. The application is a continuation—inpart—of the U.S. application Ser. No. 08/119,610 filed Sep. 13, 1993 forApparatus and Method for Spatial Scan Modulation of a Video Displaywhich application Ser. No. 119,610 is a continuation of U.S. Pat. No.355,461 filed May 22, 1989—now abandoned. The Notice of Allowance forthe application Ser. No. 119,610 was mailed to the applicant on Dec. 22,1994 and the issue fee has been paid. The content of this applicationSer. No. 119,610 is incorporated by reference.

DESCRIPTION OF THE PRIOR ART

[0003] As television, computer, graphics, printers, fax machines andrelated image technology develops, there is increasing emphasis onimproving the quality of created, displayed, or stored images in orderthat they appear more real and pleasing to the human observer. Two ofthe parameters which affect image quality, and therefore are subject toimprovement, are spatial and temporal resolution. Spatial resolution,simply put, is the number of image elements which are used to make up animage, normally static, and correspondingly, temporal resolution is thenumber of elements per unit time which make up an image, normallymoving. Desirable qualities of an image system, such as televisioncamera, scanner, broadcast television, computer display printer,permanent copy device, etc. will ideally include having as many elementsper image or frame and, in the case of moving images, as many elementsor frames per unit time as is economically feasible. Unfortunately,increasing the number of image elements per frame or the number offrames per second is a costly improvement. Therefore many schemes havebeen developed to improve the resolution of the image, while reducingthe number of elements used.

[0004] Other improvement systems, such as various Scanner InterpollationTechniques, improved Definition Television Systems (IDTV), AdvancedTelevision Systems (ATV), and High Definition Television Systems (HDTV),Hewlett Packard's Laser Printer Resolution Enhancement System, and otherImage Enhancers typically operate to increase the resolution and otherquality related aspects of image systems. Many of these systems resortto various techniques for such quality improvements, some of whichgenerate unwanted artifacts.

OBJECTS AND SUMMARY OF THE INVENTION

[0005] It is an object of this invention to provide an apparatus andmethod for improving the apparent quality of a created and/or displayedimage by altering the size, shape or position of the elements of theimage.

[0006] It is an object of this invention to allow usage of a lowresolution camera and/or recorder in a high resolution videodistribution and/or display system.

[0007] It is another object of this invention to provide an apparatusand method to alter the size, shape or position of the elements of ascanned or presented image in response to the relationship between aplurality of elements of the image.

[0008] It is a yet further object of this invention to provide a meansand method for inspecting a plurality of elements of an image todetermine the presence of a need for filling of areas between theelements.

[0009] It is yet another object of this invention to provide a means andmethod for simultaneously providing a plurality of elements of saidimage for inspection and comparison means to determine proper alteringof areas between the elements or voids created by defective or unneededelements.

[0010] It is an additional object of this invention to provide a meansand method for inspecting a certain element or location with respect toone or more surrounding or neighboring elements of an image to determinethe desirability for changing the shape, position or size of otherelements to improve the spatial and/or temporal resolution relationshipbetween the elements, which may neighbor in time or space.

[0011] It is an additional object of this invention to provide a meansand method for inspecting neighboring elements with respect to one ormore other elements of an image to determine a need for changing theshape, position or size of elements to improve the spatial or temporalresolution relationship with the other elements, especially when oneother element is defective.

[0012] It is yet still another object of this invention to provide ameans and method to improve the quality of an image by inspecting aplurality of neighboring elements to generate replication elements inresponse thereto.

[0013] It is a further object of this invention to provide a means forreplicating non-defective image elements while producing noascertainable artifacts.

[0014] It is still yet another object of the invention to apply theabove objects to any physical phenomena or signal which can berepresented as a matrix of discrete elements.

[0015] It is still another object of this invention to provide an imagecreation device utilizing the above objects.

[0016] According to an aspect of this invention, the inventive conceptsdisclosed herein show an apparatus and method for modifying the creationstorage and/or production of an image by device in response to the imagecontent thereof as carried by an image bearing signal, in order tocreate an image having apparently higher quality than normal.

[0017] The preferred embodiment of the present invention describes aneighboring element means for providing a plurality of neighboringelements, and an element replication means responsive to said pluralityof neighboring elements to selectively fill a location, which may be avoid, or artifacts, between elements, or to replace elements in thespatial or time dimensions or both. The inventive concepts disclosedherein may be utilized to improve the apparent resolution of the createdand/or displayed image spatially, temporally, or both, or to conceal, byreplication including modification, and/or creation of otherwisenon-existant, defective or unneeded image elements or artifacts. Theinvention will find considerable use in the reduction of spatial ortemporal (motion) artifacts of improved television systems like HDTV.

[0018] It will be understood that the term image as used herein is meantto apply to the creation and/or presentation of any phenomena by araster or matrix of discrete or adjoining elements, and that the rasteror matrix may be either a single one, or a given one of a plurality orsequence of rasters or matrices, for example as used in temporalportraits of such physical phenomena. The image can be visable on adisplay (such as a computer monitor or regular television set), viewableafter creation (such as a laser printer or fax machine) or otherwiseexist (such as in memory for subsequent use or on a recorder tape). Theterm image is applicable to the creation of an image (for example at alow resolution to camera); on recording (for example on a VHS machinefor HDTV transmission), on receipt (for example NTSC reception on anHDTV monitor) or otherwise.

[0019] It will be further understood that invention has application to agroup, or series of such elements, whether transmitted or stored, intime sequential or parallel arrangement or in any other form. The moreimportant aspect of the invention is the operations on the elementswhich have some spatial or temporal coherence or probability ofsimilarity. It is of lesser importance what the elements represent orhow the elements are conveyed, or of the particular nature or make up ofthe form of the elements.

[0020] It will be also understood that although the word void is used inthis specification, the invention is directed towards replicating newimage information utilizing neighboring image elements, which new imageinformation is utilized at a certain location(s). These locations mightor might not have previously had image information available therefor.The void may exist at the point of image creation, before/after storageand/or at the point of presentation. Examples of voids would includesuch things as defects, unwanted elements, improper elements, corruptedelements, valid but replacable elements, locations with no imageinformation, and/or other locations or elements which may be in questionor need for improvement. The term void is used to cover all these andsimilar situations for uniformity.

[0021] It will also be understood that although the word combination“filling in” is used in the specification and claims, the invention isdirected towards replicating an image element at a particularlocation—again, an image element which might or might not havepreviously had information available therefor. This replication includescreating, modifying, replacing, substituting, adding to, providingand/or filling in for the element at this location. This term filling inis used to cover all these and similar situations for uniformity.

[0022] It will also be understood that although the word “similarity” isused in the specification, the invention is directed towards the use ofany of the various element characteristics to determine similarity,which characteristics can be used alone or in combination thereof. Forexample, the elements both may be of the same color, but of differentbrightness; they both may be of same brightness, but of different hue;they both may be of the same luminance but of different colorsaturation; they both may be of same saturation and luminance; they bothmay be the same size; they both may have the same relationship to theirsurroundings; or otherwise be similar in some one or combination ofcharacteristics. Characteristics by which similarity can be determinedinclude color, hue, color saturation, luminance (brightness), size,detail, pattern, spacial frequency component in horizontal or verticalor diagonal or time or other dimensions, temporal frequency, content,relationship of neighboring elements, noise, and/or other externalmeasures such can be derived from a detection circuit which wouldprovide a flag or measure and/or other indication that an element orgroup of elements are suitable for processing. The element or locationwhich is being processed may or may not be a valid or erroneous element:It might even have had no image information or not previously existed.

[0023] It will also be understood that although the word “replication”is used in the specification, the invention is directed towardsmodifying, correcting, improving, substituting for, adding to, replacingor otherwise processing the image so as to provide for an overall, morepleasing or apparently higher quality image. As previously set forth theword “fill in” is used in this specification for similar attributes asreplication.

[0024] It should be understoood that the word “surrounding” is used inthis specification to describe elements which have some relationship toother image elements, be the relationship spacial or temporal. The wordsurrounding could include elements which are neighboring on one sidethereof, neighboring on all sides, adjacent thereto, spaced fromdiagonally with intervening elements between (such as in a interlacedfield scan device wherein alternating fields are paired), immediatelyadjacent or spaced elements which have a statistical ability of beingsimilar, or merely elements that have a greater than minimal statisticalprobability of being similar. However, in respect to this latter it ispreferred that the percentage of similarity, is over 50%.

[0025] It should be understood that the invention has application duringthe creation of an image (for example at a video camera), at the storageof an image (for example before or after a video tape or disk) and/or inthe production of an image (for example a video monitor).

[0026] The ojects and features of the invention will be apparent to oneskilled in the art from a consideration of the following description andclaims, when read in conjunction with the accompanying drawings inwhich:

BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The structure, operation, and advantages of the presentlydisclosed preferred embodiment of the invention will become apparentwhen consideration of the following description taken in conjunctionwith the accompanying drawings wherein:

[0028]FIG. 1 is a block diagram of a typical prior art image generatingdevice.

[0029]FIG. 2 is a block diagram of the device of FIG. 1 including thepresent invention.

[0030]FIG. 3 shows a modification of the drawings of FIG. 2 wherein thespatial scan modulator is

[0031]FIG. 4 shows a typical prior art scanned image display with aportion expanded for clarity.

[0032]FIG. 5 shows a typical scanned image display as would occur withthe use of the present invention, with a portion expanded for clarity.

[0033]FIG. 6 shows a detailed diagram of a given scan line of an imagein prior art form and in improved form resulting from the presentinvention.

[0034]FIG. 7 shows a group of nine neighboring image elements which maybe operated on by the present invention.

[0035]FIG. 8 shows a diagram of the present invention including the useof a ranking means.

[0036]FIG. 9 shows a diagram of 15 elements taken from scan line of 5sucessive image fields.

[0037]FIG. 10 shows a detailed diagram of an embodiment of a neighboringmeans for a image display.

[0038]FIG. 11 shows a detailed diagram of an embodiment of a elementcalculator means.

[0039]FIG. 12 shows an alternate embodiment of a fill and D-A converter.

DETAILED DESCRIPTION OF THE INVENTION

[0040]FIG. 1 is a block diagram of a prior art image display device 1 awhich may be improved by the addition of the present invention. Theinvention is set forth in its preferred embodiment in primary referenceto a image display, in this specification a video display. However, thisdevice may be any of those which are well known in the art which create,store and/or convey physical phenomena by use of ordered groups ofelements, such as various matrix and raster type displays or hard copygenerating display devices such as various cameras, scanners,televisions, laser printers, fax machines and the like. For the purposeof the present description and explanation it will be primarilypresented with the device 1 a that is a raster scanned display such asan electron beam scanned CRT or a laser scanned photo sensitive surfaceoperating in a progressive (non-interlaced) mode. This device may alsooperate in interlaced fashion as will be apparent from the presentdisclosure and in other creation, recording and presentation forms.However for the purpose of the present explanation of the invention, byway of example, it will be assumed that it is a video display operatingin a progressive mode.

[0041] This display device normally contains an amp 2 a which receives,clamps, amplifies and couples the signal to the display element 6 a,which is most commonly a CRT or modulated laser but which also could bemodulated LED's or LCD's or other image creation device.

[0042] The device shown is a video device also contained in the device 1a is a sync circuit 3 a which receives the signal, strips the compositesync therefrom, separates the composite sync into H and V components andcouples these components to the H scan 4 a and V scan 5 a circuitsrespectively. The scan circuits 4 a and 5 a provide the scanning controlof the display element 6 a, for example by providing the ramp drivewaveforms to the CRT yoke or displacing the laser beam, or modulatingthe imaging devices suitable modifications would have to be made for animage creation device like a camera.

[0043] It will be recognized by one skilled in the art that the deviceshown by way of simple example in FIG. 1 has many practical variationswhich are commonly used. The single connecting lines shown betweenelements will be understood to carry multiple signals as required, andother configurations may be made. For example, the sync circuit 3 a issometimes configured to receive one or more sync waveforms separate fromthe image signal. The actual scanning process may be by alternatemethods as well, operating in progressive or interlaced fashion,including magnetic and electrostatic deflection of an electron beam suchas in a CRT or E.B. recorder, electro-optical or mechanical deflectionof a light beam such as in a laser projector, fax machine, printer orelectro-optic display device, or by address or pixel selection as withthermal, inkjet, flat panel, florescent, LED or LCD type displays orprinters. The device might also be an image creation device, whethervideo tube, CCD or otherwise. Further the processing may occur atimaging prior to recording, after recording, in or out of memory or atany other time the image information is available.

[0044] One skilled in the art will be able to utilize the presentinvention with any of the many image creation, recording, memory and/ordisplay devices which utilize ordered groups of image elements toeventually set forth image phenomena as are known in the art.

[0045] The description given herein is by way of example with primaryrespect to a video monitor. As previously mentioned, video refers to thegroups of elements irrespective of their use or form of existence orcreation and will be understood to mean such even though described withprimary respect to television elements.

[0046]FIG. 2 is a block diagram of a display device 1 b like 1 a of FIG.1, including amp 2 b corresponding to 2 a, sync circuit 3 bcorresponding to 3 a, H scan circuit 4 b corresponding to 4 a, V scancircuit 5 b corresponding to 5 a and display element 6 b correspondingto 6 a. FIG. 2 also shows the spatial scan modulator 7 as an embodimentof the present invention containing neighboring element means 8responsive to the signal to provide a plurality of neighboring elementsto the element replication means 9. Neighboring element means 8 may alsobe responsive to element replication means 9, as will be discussed withrespect to FIGS. 10 and 11. The neighboring element means 8 alsoprovides a delayed version of the signal of the amp 2 b and sync circuit3 b in order to make up for processing delays as will become apparent toone skilled in the art from the discussion below.

[0047] Delay will be understood to encompass deriving different elementsin time and/or space by a variety of well known means, including thespecific example of delaying a time sequential series of image elements.

[0048] The element replication means 9 provides a V fill signal to the Vscan circuit 5 b in response to the plurality of neighboring elementsfrom 8 in order to cause a vertical filling or dithering of the elementscan or addressing at the proper times. The spatial scan modulator 7therefore operates to determine where voids or elements to be replicatedexist between elements or are created between or on elements bydefective elements, and how these voids (as previously defined) are tobe filled in in the image. This determination made by an inspection of aplurality of neighboring elements, and then causing the creation and/ordisplay device to replicate, fill, replace, or otherwise modify theappropriate location in response to the fill signal. Shown as optionalis an H fill signal provided by 9 and coupled to 4 b to cause horizontalfilling, and a further optional video fill provided by 9 and coupled to2 b to cause modulation or filling. The device can also be utilized toreplicate non-defective image elements with no ascertainable artifacts(again the term void covers all situations).

[0049] As with the various display and creation devices known in theart, the fill signals (as previously defined) provided by 9 will take ondifferent connections to the devices 2 b, 4 b and 5 b, resulting fromthe differences in those devices.

[0050] It is the object of the fill signals from 9 to cause theappropriate filling of locations in the image by whatever means issuitable for the particular imaging device being used, whether duringcreation, recording, and/or presentation of the image. These locationswhich may be voids in the image which may be filled with all or aportion of a element, or all portions of combinations of elements, orotherwise as will be described in greater detail later, especially withrespect to the specific video display embodiment FIGS. 10 and 12.

[0051] In the present example, the voids or element locations are filledby slightly dithering the example electron, light beam or other imagingdevice away from its normal scan position by manipulation of the sweepcircuits of the scan circuitry during creation and/or presentation ofthe image element.

[0052] For a light beam type device the voids may be filled bydithering, modulating or focusing the beam. For addressing type devicesthe dithering may be accomplished by manipulating addressing and fillingas may be accomplished by such dithering. In all devices, filling may becaused by modification of existing elements.

[0053] For the device described by way of example, the V fill signalwill cause the electron, light beam, imaging or creation device to bemodulated vertically, either upward, downward or sideways or both tocause given scan line to become wider in an upward, downward, sidewaysand/or all directions. In a video circuit this can be simplyaccomplished by adding or subtracting a small amount of high frequencypulsating current in the sweep or scanning driver circuit in a videocircuit, thus broadening the electron or laser beam slightly upward ordownward from its normal position on the face of the CRT orphotosensitive surface. Similarly, the H fill signal can cause a slighthorizontal displacement of the beam position by adding or subtracting asmall amount of high frequency current from the H sweep or scanningdriver circuit. In order to accomplish diagonal or angular displacementof the electron beam, a modulation of both H and V deflection circuitsmay be made. Such modulation of the electron beam position is relativelyeasy to accomplish in terms of circuitry requiring only a bidirectionalcurrent source, or a pair of unidirectional current sources, which areswitched on and off at high frequency rate in response to the fillsignals, and add or subtract current in the sweep or scanning circuit,thereby slightly modulating the current flowing in the driver and thusaltering the electron beam deflection. The operation thus causes adither of the beam modulation, which will be explained in greater detailwith respect to FIG. 5.

[0054] In alternate creation/presentation circuits the signals to/fromvarious circuits would be appropriately modified.

[0055] The beam modulation of any image conveying beam such as electronor light can also be achieved by changing the deflection, intensity,shape, duration, focus or astigmatism of the beam, and/or scanning,thereby changing the spot size and/or shape. The modulation of theconveying beam may be caused to take place in various directions aswell, for example in diagonal directions, in response to the pixelreplication means 9. Such improvements will be understood and may bemade by one skilled in the art in view of the present teachings.

[0056] The art of changing electron beam deflection by small amounts hasbeen previously practiced, for the purpose of geometric scan correction.In general the exact method of effecting the modulation of the displaywill be determined by the nature of the display printing, or imagingdevice; however, one skilled in the art will be able to devise propercircuitry to practice the present invention for a given desired type ofcreation and/or display device, in view of the teachings herein.

[0057] The fill signal may be utilized to generate the desired elements(which may be including additional elements replication) on the inputsignal in response to 9, with or without scan modulation, thus providingelements to fill in the desired blank areas of the image (which may beblank as in voids, unneeded, unwanted, or otherwise).

[0058] Filling may be accomplished by simply adding the fill signal tothe signal in a monitor device so that the electron beam is caused toilluminate the CRT phosphor in response to the fill signal as well asthe signal.

[0059] Illumination may be caused to occur in the absence of a signalgenerated illumination, or may cause the signal generated illuminationto be modified, such as by being increased or decreased. The image fillsignal may also be utilized to perform other image functions as willbecome apparent, such as reducing the bandwidth of the signal, or bychanging the element or spot size for example by defocusing the electronbeam, repeating a displayed element, or generating a new element(especially in non-direct scanning devices).

[0060] New elements used for filling of voids including the substitutionor replication of existing elements may be comprised of all or a portionof a element or group of elements.

[0061]FIG. 3 shows the same elements as in FIG. 2, however theneighboring element means 8 is coupled to amp 2 b to receive the inputsignal, rather than directly to the input, as in FIG. 2. Several otherembodiments relating to coupling and arrangement of elements will bepossible, as will be apparent to one skilled in the art, and areintended to be considered within the scope of the invention as hereindescribed and claimed.

[0062]FIG. 4a shows a typical scanned image 10 a (whether creationand/or reproduction) with a small portion 11 a which is expanded forclarity in 4 b. The small portion 11 a shows four scan lines 12 a, 13 a,and 15 a. The scan lines may be considered consecutive for the purposeof the present explanation, but may also be alternate as in aninterlaced display, or otherwise non-consecutive. The scan lines areshown thicker during the higher illuminated portion, the thicknessrepresenting the width or intensity of the scanning beam, with thenarrow line representing the scan track normally followed by the beam,as is common for laser beam projection and recording devices and CRTs.Also this scan line is representative of what exists in alternateimaging devices.

[0063] With more modern creation and/or presentation devices themodification is more theoretical-occuring in electronic form duringand/or after creation instead of during presentation.

[0064] Scan line 12 a has seven illuminated element points 16 a through16 g identified for clarity. As with many systems, the individualelements typically blend together when sequentially aligned along a scanline, due to both the width of the beam, and the limited bandwidth ofthe video amplifiers in the device, giving rise to the continuoushighlighted areas shown. It will be understood that the individualelements may also represent a matrix display rather than a scanneddisplay. The four scan lines show a diagonal bright area which can beseen to take on a rather stair stepped appearance. The stair steppingand the space between the scanning lines make up picture artifacts thatviewers find objectional in viewing the image, whether created duringcreation and/or presentation.

[0065] It is one object of the present invention to fill in the voids ofthe stair stepping and voids between consecutive scanning lines, forexample by modulating the scanning of the electron beam in a videodevice, thus reducing these objectional artifacts.

[0066]FIG. 5a shows the same display 10 b as 10 a in FIG. 4a, withexpanded portion 11 b corresponding to 11 a, and showing scan lines 12b, 13 b, 14 b, and 15 b corresponding to 12 a, 13 a, 14 a, and 15 arespectively. Again the creation/alteration presentation circuits wouldbe appropriately modified.

[0067] Illuminated elements 17 a-g corresponding to 16 a-g are shown aswell as illuminated pixels 18 a-e. For each element which is to bepartially modulated during creation or during production, an up or downarrow is shown indicating the direction(s) of modulation. For example,element 17 b is modulated downward to fill the void between 17 b and 18a. Likewise elements 17 c-g are all modulated downward. Elements 18 aand 18 b are modulated upward to fill the same void and 18 c-e aremodulate both upward and downward. One skilled in the art will recognizethat the image of FIG. 5 has a higher apparent resolution than that ofFIG. 4, the improvement being a result of the filling of voids (even ifpreviously occupied by valid elements). This modulation can occur in theoriginal image scanning circuitry, modifying the element signal duringor after storage, or otherwise. The same type of image modificationwould also occur in alternate imaging devices such as scanners, laserprinters or fax machines.

[0068] With respect to FIG. 2, when element 18 a is being scanned, theneighboring element means 8 would provide the elements adjacent to 18 a,including 17 b, 17 c 17 d, and 18 b, for inspection by elementreplication means 9. The element replication means 9 will determine thatthere is a void between 17 c and 18 a which should be filled, thuscausing 18 a to be modulated upward into the void. The void may beactual or artificial (as when the device ignores valid elements).

[0069] It can be seen from inspection of FIG. 5 that a good portion ofthe voids between scanning lines or matrix elements, as well as aportion of the stair stepped edge void have been filled in. Element 18 fis shown as being a defective element which has been replaced thuscausing filling of the void which otherwise would have been present tobe filled.

[0070] It may be noted that an object of the present invention is toprovide a method of filling voids without restriction to the nature ofhow such voids arise, although it will be appreciated that the nature ofsuch filling may very will be optimized in response to the nature of thevoid and that the voids might discard valid image elements at theinvolved location. For example the filling of voids between elements maybe performed differently than the filling of voids created by defectiveelements. As another example, assuming 18 f were defective due to adefective video element, the element may be replaced to cure the defect.If, on the other hand, the element for 18 f were not defective byitself, the artifacts produced by a defective florescent element, thelocation could be filled by lighting, or increasing the lighting of theneighboring elements. Further, the invention may be utilized toreplicate valid, non-defective image elements due to the probability ofsimilarity. This allows continual processing of a signal through acircuit without on/off switching while producing minimal artifacts.

[0071]FIG. 6 shows a typical prior art scan line 19, corresponding to 13a of FIG. 4b, having an illuminated section of 6 elements shown as awider portion of the line. The elements may be created in any known waysuch as by scanning light or electron beams, LED's, LCD's or laserbeams, by spatial addressing an imaging device or by signal alteration.

[0072] As with the previous example of FIGS. 4 and 5, the individualelements have blended together gaining a continuous illuminated section.The presented scan line 20 shows the same line as 19, except that it hasbeen spatially modulated in accord with the present invention thuscorresponding to 13 b of FIG. 5b. The track of the beam is shown in 20as a sinusoidal path, or spatial modulation, which deviates from thestraight scanning line of the prior art. This same result could beprovided by alternate means.

[0073] Note that for the left two elements the deviation takes placeboth above and below the line. In the preferred embodiment of theinvention the electron beam or memory laser path is such that the trackpitch, the spacing between the points where the track crosses the normalscan line, is less than the electron or reproduction beam width.Therefore the electron or beam width will illuminate a solid area. Theilluminated area created by the beam path shown in 20 is shown as asolid are in 21. It will be noted that the ability to fill in chosendirections only, such as only up to 21, is an important feature.

[0074] A note should be made about the relative brightness of thespatially modulated scan of 21 versus the area of the prior art scan 19.The brightness of a given area is a function of the flux density of theelectron, light beam or image signal striking that area, that is a givennumber of electrons or photons or other image creating energy will tendto provide a given number of photons of visible light or particles ofdye or pigment, independent of the area which it strikes. For a givenelectron or beam intensity, the area of 21 will therefore appear dimmerin terms of visible photons or particles per unit area than the area of19. If a given spatial scan modulated area of the image is large enoughto be resolved as a distinct element, the viewer may notice thisdecreased intensity. In order to overcome this decreased brightness, itwill be desirable to increase the intensity of the image creating beam,therefore restoring proper brightness in those areas where the spatialmodulation is occurring. The brightness increase will typically be afunction of the amount of modulation. For example while the beam ismodulated only upward for the left two elements of 20, a first givenincrease should be made, and for the remaining elements where the beamis modulated both up and down with a second, higher, increase made. Thisincrease in beam intensity can be made in response to elementreplication means 9 shown in FIG. 2, via the fill signal. It may also bedesirable to utilize the fill signal to increase the number ofilluminated elements, in conjunction with the spatial scan modulation.For example, the illuminated area of line 20 (13 b of FIG. 5) can betuned on one element early in order to further reduce the stair steppingof the diagonal edge. This may be accomplished by time or spatialmodulation of the image creating beam.

[0075] Alternate imaging devices would similarly operate, albeit withadaptions for their special properties. For expample, with a LED or LCDimaging device (such as a printer), modifications of the brightness ofsuccessive lines of image information and/or the modulation of the imageelements in what would otherwide be spaces between lines would be oneway to produce the image improvement of the present invention.

[0076]FIG. 7 shows a group of 9 elements which are located on 3 scanlines of a raster or alternatively in 9 matrix locations. These nineelements are provided by the neighboring element means and may beutilized by a element replication circuit in order to determine ifspatial scan modulation or other filling should be enabled. In thefollowing discussion it will be assumed that element X (the centralelement) is defective or wanted to be replaced, as is known from errordetecting circuitry or as determined by the element replication 26, oris otherwised to be replicated. It will be assumed for the purpose ofthe present example that element X in the center of the 9 elements, isthe element about to be displayed or created by the display element.Scanning will be assumed to be from left to right, top to bottom.However it will be understood that the present explanation will applyequally well to any image creating and/or displaying device or systemindependent of the manner in which the image elements are scanned orplaced as well as to any point of the image for example element A inFIG. 7 could be processed.

[0077] For the described scan, elements A-D will have occurred in timebefore X, and elements E-H occur in time after X. The 9 elements are allmade simultaneously available to the elements replication by theneighboring elements means (8 of FIG. 2). The neighboring elements meansin the preferred embodiment is made up of memories, delay lines, shiftregisters or other delay devices which are well known to one skilled inthe art, which allow all nine of the elements to be presentsimultaneously. A different number of elements could also be present.

[0078] A device which can be utilized for the neighboring elements meansfunction is described, with respect to FIGS. 9 and 10, in U.S. Pat. No.4,573,070 issued Feb. 25, 1986. Other arrangements and circuits toperform this function will be apparent to one skilled in the art fromthe present disclosure, for example, retrieving elements from RAM as inmatrix displays. In order that elements X can be the about to becurrently displayed elements, it is necessary that neighboring elementsmeans make elements available to the display device, which is shown bythe connection from 8 to 2 b and 3 b of FIG. 2, which replaces the videoinput to the display device.

[0079]FIG. 7 shows eight possible replication direction differenceswhich can determine the direction of replication for an element X. Thiselement X could be defective, non-existant merely unnecessary, ordisposable (even if valid)—the latter most particularly in respect to anunswitched, constantly operating device. The major factor is the desireto have element, replication, or substitution. The eight differences arerepresented by eight lines P-W. Line P represents the difference ofelements A and H, Q of elements B and G, T of elements B and E, etc.

[0080] Element X may well be replicated from any direction, includingthose of the third dimension, which would represent a frame to frame ortime direction, or a combination of time and spatial dimensions.Replication in the time dimension is useful in improving motionsartifacts. Time replication is accomplished by using delays of onepicture period (field or frame in a monitor device) or more to provideelements in the time axis, which may be used to fill temporal voids. Itis particularly useful in a video imaging device.

[0081] Element X will thus modulated in response to elements which arepresent in field or frames other than the one containing X. U.S. Pat.No. 4,573,070 which is incorporated herein by reference describes morefully various embodiments of neighboring elements means which may besuitable for use in this fashion, and particular FIGS. 13, 15 and 16 andthe description thereof teach detection of a defective element X.Alternatively, the detection of a defective element X may be performedby various defect or error checking circuitry as is well known in theart.

[0082] It will be immediately recognized that by comparing elements onthe opposite sides of X, it is possible to determine which pair ofsurrounding or neighboring elements which are most similar. The mostsimilar pair thus represents the pair of image elements most likely toprovide the least noticeable replication value for an element X, andalso indicate the direction(s) of modulation of X to fill voids adjacentthereto. For example if A-B=9, B-G=7, C-F=8, and D-E=3, the preferredpair of elements for replicating X would be pairs D and E and eithercould be chosen for replication. Alternatively, a normalized combinationof the two pairs such as an average can be used.

[0083] This logic holds true if in fact X is related to elements on twoopposite sides. However X may be related only to the elements in thecorners, that is to elements A, B, and D (upper left), B, C, and E(upper right), E, H, and G (lower right), or G, H, and D (lower left).The logic conditions shown below thus preferably takes all eightconditions into account, selecting the lowest difference pair ofopposite or corner elements to determine the direction(s) of modulationof X, to fill voids adjacent to X, or giving the direction or pair ofelements most likely to provide the least noticeable replication valuefor an element X in FIG. 7.

[0084] For the purpose of the present description and the purpose ofexplanation, it will be assumed that only the two dimensions, and thereplication directions indicated by the 7 differences are to beconsidered.

[0085] A group of similar elements is easily implemented by logicoperations as may be utilized to determine which if any of the 8 spatialreplication directions should be enabled for a given elements X. Logicoperations that may be used to enable the modulations according to thefollowing table include: MODU- LATION DIRECTION CONDITION VERT. B-G <A-H or C-F or D-E or B-E or E-G or G-D or D-B HORIZ. D-E < B-G or A-H orC-F or B-E or E-G or G-D or D-B R.DIAG. C-F < B-G or A-H or D-F or B-Eor E-G or G-D or D-B L.DIAG. A-H < B-G or C-F or D-E or B-E or E-G orG-D or D-B U.LEFT D-B < B-E or E-G or G-D or B-G or A-H or C-F or D-EU.RIGHT B-E < E-G or G-D or D-B or B-G or A-H or C-F or D-E L.LEFT E-G <G-D or D-B or B-E or B-G or A-H or C-F or D-E L.RIGHT G-D < D-B or B-Eor E-G or B-G or A-H or C-F or D-E

[0086]FIG. 8 shows by way of example an embodiment of the presentinvention which may be utilized with image elements. The FIG. 8embodiment contains a neighboring element means 25 similar to 8 of FIG.2, and an element replication means 26 similar to 9 of FIG. 2.

[0087] The element replication 26 contains a rank logic means 28 whichcooperates to inspect the 9 neighboring elements A-H and X presented onlines 30A-H and X respectively, and provides a fill signal on line 32and replaces signals on line 36 respectively, which device operates tocontrol the spatial modulation of element X. It should be noted that inFIGS. 8 and 12 the fill and replace lines are shown as single lines forclarity, and further that the connection which provides element X to thedisplay device as well as the video fill connection to the display ampare not shown for clarity. One skilled the art will however recognizethe need for, and be able to provide the proper coupling of, element Xand element fill and replace functions to the image device via multipleconnections and couplings with the element replication as appropriatefor a given application.

[0088] As will be apparent from the present teachings, many of thefunctions of the present invention can be implemented with various formsof hardware including ASICS, programmable logic and analog or opticalcircuitry and software running on any of the various well knownplatforms. For example, microprocessors with suitable software may beutilized. As a further example, a read only memory may be utilized. Inparticular, a ROM or programmable logic would be well suited toimplement part or all of the elements replication means 26.

[0089] The determination of which neighboring elements are related toelement X is a ranking process, which is described in one form in somedetail in the aforementioned U.S. Pat. No. 4,573,070 with respect tovideo noise reduction. The '070 patent does describe and claim thereplication of defective elements, for example at column 4 line 49 et.seq. and column 8 line 45 et. seq. The ranking circuitry shown in FIGS.15 and 16 of the '070 patent can be utilized for the rank logic means of27 of FIG. 8 of the present invention, but the rank logic means of thepresent disclosure has been found to provide superior performance whenutilized for applications to replace defective or unneeded elements. Therank logic means 27 operates in response to the elements A-H and ifneeded X provided via 30A-H and 30X to determine each of elements A-Hrelative closeness to all the others. For example if elements C-F weredetermined to be closest to each other, then signifying a rank of 0(that no other element pairs are closer) would be output on line 31R. If2 other elements were closer than elements C and F, a 2 would be outputon line 31R. Each element pair will be ranked by 27, with that pair'srespective rank provided on line 31 P-W. The fill logic means operatesto select a given number, for example 2 or 4 of those elements which areclosest and identifies them. The logic equations given above for theexample of FIG. 7 are then used to determine the modulation and/orreplication directions to be enabled. This ranking may occur bycomparing all groups for similarity (parallel processing) or bycomparing each group to the next group sequentially retaining the mostsimilar (serial processing). This latter is more efficient.

[0090] Further the rank logic means may also operate to rank elements inrespect to their dissimilarity, thus not utilizing elements that do notmeet a minimum criteria for similarity ranking. This recognizes thatelements that are known to be largely dissimilar do not have to beranked for similarity. This provides considerable saving (of mostparticularly processing time) and circuit complexity since thecomplexity of calculations increases exponentially as to the number ofelements utilized. Preferably therefore the ranking of elements woudl bea multi-step process: first discard dissimilar groups of elements notsufficiently similar in respect to each other and/or other sets so as tonot warrant subsequent processing providing at least one set remains andthen second ranking of the groups of similar elements. A threshold toestablish similarity is particularly effective in the initial step.

[0091] Note that more than two elements may be compared in a group, forexample groups of 3 or more may be compared. For example, group ABD,BCE, EHG, and GFD may be compared to determine the total difference, forexample A-B+A-D+B-D as the difference for ABD.

[0092] As an example, assume that the element pairs have the followingranks: P = 0 Q = 1 R = 7 S = 2 T = 6 U = 5 V = 4 W = 3

[0093] From the above chart it can be seen that pairs P, Q, and S areclosest thus indicating element X is most likely related to a diagonalfrom element A to H. The three closest elements will be identified as Aand H, and of the above logic equations the following will be satisfied:

[0094] VERT

[0095] HORIZ

[0096] L. DIAG

[0097] These equations being satisfied, the modulate VERT signal, andmodulate HORIZ signal, and modulate LEFT DIAGONAL signal will beactivated. As a result of the above analysis, the void between elementsA, B, D, and X and E, G, H, and X would be caused to be replicated byspatial modulation of elements X if X were not defective. Alternatively,the voids and/or element X could be replicated by elements A, B, D, andE, G, and H, or combinations thereof respectively. As an improvement tothe spatial modulation, the amplitude or spatial intensity of themodulation may be changed in response to the ranking of the neighboringelements. For example, if the difference between X and A is small, alarge amplitude of modulation is used; and if the difference were large,a small modulation used. The amount of modulation is therefore caused tovary in response to the difference, in either spatial or temporalembodiments. Of course, intensity or other atribute modulation may alsobe provided as well, as previously discussed. Additionally, if element Xis defective or unneeded, it may be deleted, thus creating a void, andthe void may be replicated with a combination of those element pairshaving the closest values, in the above example, pair A-H or Pairs A-Hand B-G. For a continually operating device, element X could bereplicated even if not defective.

[0098] It is important to note that by ranking the various element pairdifferences P-W that a very accurate prediction of the value of X may bemade in the event X is defective. By inspecting the differences andtheir ranking, the individual surrounding elements, in space and time,which are most closely associated with X may be determined with a highprobability. A combination of a plurality of those most closelyassociated elements may then be utilized to replace element X, to fillany voids near a good or defective X, or even to totally replace a validelement X (which would treat X as a void irrespective of its truenature).

[0099] Such combinations which have proven suitable for use include anyof the various averages which are well known, including arithmetic andgeometric averages, means, medians, weighted combinations, spatial andtemporally filtered combinations and various interpolations and curvefittings. It has also been found that selecting a single element fromthe group of most closely associated elements to use for fill orreplication works quite well. In particular, choosing a median valueelement of the most similar group has been shown to offer quite goodperformance. In particular, choosing the two most closely associatedgroups of elements, and choosing one of the two median elements of thegroups, the one closest to the average of the four, has been found togive quite good performance.

[0100] In summary then, the preferred operation begins by selecting alocation for image enhancement. This location may be anywhere in theimage including along the edges and corners thereof. Once the locationis ascertained, the plurality of groups of two or more elements having arelationship to the location are ascertained. These groups may beadjoining, neighboring, having a theoretical similarity, and may bepresent with a time or space variable or combination thereof. After thegroups are ascertained, each image element in the group is compared tothe other elements in the same group to determine the similarity or howclosely the individual elements within each group match. Note that thisis possible for a single element to be in more than one group. Thissimilarity is preferably typically determined by the absolute value of Aminus B or similar function (to increase operating speed and reducecircuit/software complexity). This comparison could include apreliminary dissimilarity threshold which would in operation not processelements which are significantly dissimilar to other sets or to apredetermined theoretical or percentage value of difference. Thisdissimilarity threshold could be by ranking of the groups of elements orby comparison of the differences between image elements in each group.

[0101] Once the similarity within each group has been ascertained, thesimilarity of each group is compared to the similarity of all the othergroups to ascertain which groups have the most closely matching imageelements or most similar image elements. This can be accomplished bysequentially comparing the similarity of each group to a subsequentgroup, discarding the most dissimilar before repeating the process, byranking the groups outright according to their similarity, or otherwise.

[0102] Once the most similar group, or groups, have been ascertained,then one or more elements within the most similar group are used togenerate a replication, value for use at the particular location. Theimage element(s) chosen can be one of the most similar pair, an averageof the most similar pair, an average of the image elements in a group ofclosely related image elements, an average of the most similar of theimage elements within groups of associated image elements, an average ofall of the image elements within the associated groups or otherwise.Further, the image elements can be replacing an otherwise valid imageelement, replacing a defective image element, or filling in empty spacesin time or space around the location. All this is separate of the needto determine whether or not whatever may exist at the particularlocation is defective, or missing since replication provides very goodestimate of the noise free value of a given image element at aparticular location.

[0103]FIG. 9 shows an element configuration which results from oneembodiment of the present invention. From the above description it canbe seen that it will be relatively easy to configure the invention tooperate in time to generate new lines, rows or new fields or frames ofvideo. Such a configuration may be envisioned by assuming that elementsA, B, and C are from a first field (or row or line); D, X, and E from athird, generated field (or row or line); and elementss F, G, and H froma fifth field (or scan line). Intermediate fields or rows or lines withelements I J K and L M N can also be utilized. Such a system is shown inFIG. 9 where elements A-C, I-K, L-M are taken from the incoming videosignal and elements D, E, and X are fill or synthesized elements whichare generated in response to the operation of the elements which aregenerated in response to the operation of the elements replication.

[0104] The new element X would be the value calculated for element X asdescribed above, which is stored in a field or other memory matrix anddisplayed in the prior time or space sequence by the display element.Alternately, the new elements could be a calculated value derived from aplurality of the elements, as will be discussed with respect to FIG. 12.

[0105]FIG. 10 shows the preferred embodiment of the neighboring elementmeans 25 of FIG. 8 in greater detail in a video application. Input videois coupled to an A-D converter 29 where it is digitized and output as adigital video data stream. The input video is also coupled to a syncstripper and PLL circuit 30, which provides all of the required clockand timing signals for the elements of 25 and also the circuit of FIG.11. The connection of the various clock and timing signals are not shownhere for clarity. The A-D converter 29, and sync stripper and PLL 30 arewill know in the art, and are commonly found in such devices as timebasecorrectors and other video processing devices. One skilled in the artwill be able to utilize any of a number of well known circuits and I.C.sto implement 29 and 30. For example, the AD9502 Hybrid Video Digitizer,provided by Analog Devices of Norwood, Mass., Provides all of the PLLand digitizing functions of 29 and 30. The digitized video is output ata fixed number of samples per line depending on the particular versionof the AD9502. Digital Delays 31, 32 and 33 are coupled to the digitalvideo output from 29 in order to provide a plurality, in this example 9,of delayed versions of the digital video signal. 1 H delay I.C.s,suitable for 31 and 32, are available, such as the N.E.C. UPD41101, and1 field delay I.C.s suitable for 31, are also available, such as theN.E.C. UPD41221. Application notes on the use of these I.C.s areavailable form N.E.C. Electronics Inc. in Mountain View, Calif. A 1element delay I.C. suitable for 33 is the 74HCT374 Octal Latch,available from multiple sources. At any instant, nine different elementswill then be present at the output.

[0106] Referring to FIG. 10, one skilled in the art will note that ifdelays 31 and 32 are 1 scan line (1H) and delays 33 are 1 elements each,then the elements labeled A-H and X in FIG. 7 will be present at theright side output of 25. In this example, the display of FIG. 7 isassumed to be either a progressive display, or an interlaced displaywithout the second field shown.

[0107] Assuming, for another example, that FIG. 7 shows scan lines 22and 24 from an earlier field of interlaced scanning, and line 23 is froma present field of scanning, one skilled in the are will recognize thatif delay 31 is one field, the output from 25 will correspond to theelements depicted by FIG. 7.

[0108] One skilled in the art will realize that the one field delay 31will be required to make available at H, the previous field scanningline above the present field scanning line available at E. It should benoted that in many interlaced systems, the actual length of the delaywill vary by one line from field to field, depending on whether the evenor odd field is being delayed. In the use of the N.E.C. UPD41221 I.C.,the proper alignment of input and delayed video is simply accomplishedby use of the increment (INC) and decrement (DEC) controls. One skilledin the art will however, be able to construct such a delay, as is wellknown in the art. When used with matrix or other types of non timescanned video, RAM memories may be substituted as will be known fromthese teachings.

[0109] Of course the invention may be utilized with digital data, suchas in D1 or D2 digital video or with fax, modem or laser printer data aswill be apparent to one of ordinary skill in the art from the teachingsherein. The invention is particularly useful with compressed data, andprovides considerable image enhancement and reduction of both randomnoise and defective elements noise for such digital images, especiallyfor JPEG and MPEG compression systems.

[0110] In order to make elements X available in analog form for use by adisplay element, as shown in FIG. 2, digital elements X is displayed bya compensating delay 34, and then coupled to a digital to analogconverter and video fill circuit 35, where it is converted to an analogsignal. The N.E.C. UPD41101, or a combination of 74LS374 I.C. s, can beused for 34. The Analog Device s HDG-0805 Hybrid Video D-A is a suitablepart for the D-A function of 35. In addition, the HDG-0805 has a 10%bright control which increases the video signal level by 10%. This 10%bright control may be coupled to the video fill signal from the elementsreplication. Alternatively, a separate video gain circuit may beutilized to control the video fill signal. Such gain circuits are willknown in the art. In the preferred embodiment, the analog elements X maybe coupled directly to the CRT. The D-A converter also maybe caused tovary the intensity of analog elements X in response to a video fillsignal coupled from the fill logic circuit 28 of FIG. 11. The purpose ofthe compensating delay 34 is to ensure that element X is delayed by theproper time with respect to the element replication generation of fillsignals. The compensating delay may also be used in a temporalembodiment of the present invention to provide element X at the propertime (i.e. to place element X in the proper location in the displayedelement sequence) for display in response to the element replicationmeans. One skill in the art will recognize that it will be possible tocombine delay 34 with the delays 31-33 which provide the neighboringelements. For example, if 34 is to have a delay of 1 element, thenelement D can be coupled directly to 35, eliminating 34.

[0111]FIG. 11 shows the preferred embodiment of 26, the elementreplication means, of FIG. 8. The rank logic means 27 of FIG. 11receives element A-H and computes pair differences P-W from theneighboring element means 25 of FIG. 10. Each element A-H is compared toits corresponding partner to determine the relative difference by adifference determining circuit. Each difference is then compared to allof the other differences by a difference comparison circuits 37. For the8 elements A-H there are then 8 difference determining circuits and 28difference comparison circuits 37. Any pair(s) that has a dissimilarityto the other pairs are preferably disregarded in subsequent processing.The 7 outputs of the difference comparison circuits which are responsiveto a given elements pair(s) difference comparisons are coupled to anindividual ranking circuit 38, which in the preferred embodiment ischosen to be a PROM. The PROM outputs a binary number which correspondsto the number of other differences a given difference is larger than.For example, if the difference is larger than 3 other differences, theoutput of the P rank P PROM would be 3.

[0112] The rank values from each ranking PROM are coupled to the filllogic circuit 28, which in the preferred embodiment is made up of PROMI.C.s. The fill or replication logic circuit 29 generates the previouslydiscussed replication signals and/or chooses appropriate matchingelements in response to the 8 rank values. The replication signals arethen coupled to replication circuit of 35 in FIG. 10, as well as thedisplay element, as previously described.

[0113]FIG. 12 shows as an alternate embodiment of the video fill and D-Aconverter 35 of FIG. 10 in applications depicted by FIG. 9. The functionof the preferred embodiment of FIG. 12 is to generate a fill elementwhich is similar or equivalent to element X. This embodiment of FIG. 12generates a fill element, for use as element X of FIG. 7 or 9, inresponse to the video fill or replication signal from FIG. 11. The fillelement may be a combination of a plurality of elements. The fillelements may be caused to be only a single elements, or a combination ofelements in response to the video fill signal. The embodiment of FIG. 12operates to generate a combination of elements in response to the videofill signal, and combine that combination with voids around element X inthe adder. The combination is then converted to analog by the D-Aconverter. Alternatively, a replication from element X may also begenerated by the combiner, in response to the video fill signal. Thereplace signal 36 causes the adder and switch to replace X with thecombination from the combiner. The normalization of the replicationcombination may also be adjusted. Various types of combinations andweighted averages can be utilized for this combination, including theuse of more than two elementss in the combination to generate the fillelements(s).

[0114] While the above preferred embodiment of the invention has beendescribed by way of example, many other embodiments may be utilized tooperate in a given video system. For example the invention may beutilized with interlace scanning systems, or with multiple channeldisplays such as RGB color displays. A matrix of less or more than thesuggested 9 elements may be utilized, which picture elements may beadjacent or non-adjacent, and may be symmetrically or non-symmetricallychosen. To one skilled in the art it will be apparent from the presentteachings that there are numerous variations, configurations andembodiments of the above described invention which variations may betailored into a specific embodiment to maximize effectiveness with aparticular display device and video system without departing from thespirit and scope of the invention as hereinafter claimed.

What is claimed:
 1. An improved signal processing circuit for filling ina void located at a position having at least partially surrounding imageelements, said circuit including providing means to provide at least twosets of surrounding image elements, similarity means to compare eachsaid set of surrounding image elements to determine the similaritybetween the elements therein respectively, and to select the mostsimilar set. and replication means to fill the void with at least oneimage element from the set of said at least two sets of surroundingimage elements which contains the most similar image elements.
 2. Thecircuit of claim 1 wherein said similarity means includes a rankingmeans for ranking each set of surrounding image elements by similarity.3. The circuit of claim 2 wherein said ranking means ranks three or moresets of surrounding image elements by similarity.
 4. The circuit ofclaim 1 wherein said replication means utilizes multiple sets ofsurrounding image elements if said multiple sets are similar to eachother.
 5. The circuit of claim 4 wherein said replication means onlyutilizes multiple sets if said multiple sets each include at least oneadjacent and/or common image element.
 6. The circuit of claim 1 whereinsaid similarity means uses the difference between image elements withineach set in its respective determination of similarity.
 7. The circuitof claim 1 wherein said replication means utilizes the average of theset of said at least two sets of surrounding image elements whichcontains the most similar image elements.
 8. The circuit of claim 1wherein said providing means provides at least one set of image elementswhich includes image elements on differing sides of the missing void. 9.The circuit of claim 8 wherein the differing sides include the oppositesides of the missing void.
 10. The circuit of claim 1 wherein saidproviding means provides at least one set of image elements withadjacent image elements.
 11. The circuit of claim 10 wherein theadjacent elements include immediately neighboring image elements. 12.The circuit of claim 1 wherein said providing means provides at leastone set of image elements extending across adjacent fields.
 13. Thecircuit of claim 1 wherein said providing means provides at least oneset of image elements extending across sequential fields.
 14. Thecircuit of claim 1 characterized by the addition of determining means todetermine if an image element is defective and removal means responsiveto said determining means to remove a defective image element to producea void.
 15. The circuit of claim 1 characterized by the addition ofremoval means to remove an image element which is not defective toproduce a void.
 16. The circuit of claim 1 wherein said providing meansprovides at least one set of image elements based on time.
 17. Thecircuit of claim 1 wherein the image elements are elements.
 18. Thecircuit of claim 3 wherein said ranking means includes a thresholddissimilarity means, said threshold dissimilarity means blocking theranking of any set of surrounding image element which is dissimilar toat least the two most similar sets.
 19. The circuit of claim 1 whereinthe image elements come from a camera for transmitting, recording orstorage and characterized in that said replication means operates on theimage elements prior to transmitting, recording or storage.
 20. Thecircuit of claim 1 wherein the image elements are transmitted, recordedor stored and characterized in that said replication means operates onthe image elements after transmitting, recording or storage.
 21. Animproved signal processing circuit for filling in a void located at aposition having at least partially surrounding image elements, saidcircuit including providing means to provide at least two sets ofsurrounding image elements, similarity means to determine the differencebetween image elements within each said set of surrounding imageelements to determine the similarity between the elements thereinrespectively, to select the most similar set, ranking means for rankingeach set of surrounding image elements by similarity, and replicationmeans to fill the void with at least one image element from the set ofsaid least two sets of surrounding image elements which contains themost similar image elements.
 22. The circuit of claim 21 wherein saidranking means ranks three or more sets of surrounding image elements bysimilarity.
 23. The circuit of claim 21 wherein said replication meansutilizes multiple sets of surrounding image elements if said multiplesets are similar to each other.
 24. The circuit of claim 23 wherein saidreplication means only utilizes multiple sets if said multiple set eachinclude at least one adjacent and/or common image element.
 25. Thecircuit of claim 21 wherein said replication means utilizes the averageof the set of said at least two sets of surrounding image elements whichcontains the most similar image elements.
 26. The circuit of claim 21wherein said providing means provides at least one set of image elementswhich includes image elements on differing sides of the void.
 27. Thecircuit of claim 26 wherein the differing sides include the oppositesides of the void.
 28. The circuit of claim 21 wherein said providingmeans provides at least one set of image elements with adjacent imageelements.
 29. The circuit of claim 28 wherein the adjacent elementsinclude immediately neighboring image elements.
 30. The circuit of claim21 wherein said providing means provides at least one set of imageelements extending across adjacent fields.
 31. The circuit of claim 21wherein said providing means provides at least one set of image elementsextending across sequential fields.
 32. The circuit of claim 21characterized by the addition of determining means to determine if animage element is defective and removal means responsive to saiddetermining means to remove a defective image element to produce a void.33. The circuit of claim 21 wherein said providing means provides atleast one set of image elements based on time.
 34. The circuit of claim21 wherein the image elements are pixels.
 35. The circuit of claim 21characterized by the addition of removal means to remove an imageelement which is not defective to produce a void.
 36. The circuit ofclaim 22 wherein said ranking means includes a threshold dissimilaritymeans, said threshold dissimilarity means blocking the ranking of anyset of surrounding image element which is dissimilar to at least themost similar set.
 37. The circuit of claim 21 wherein the image elementscome from a camera for transmitting, recording or storage andcharacterized in that said replication means operates on the imageelements prior to transmitting, recording or storage.
 38. The circuit ofclaim 21 wherein the image elements are transmitted, recorded or storedand characterized in that said replication means operates on the usageelements after transmitting, recording or storage.
 39. An improvedsignal processing circuit for filling a void located at a positionhaving at least partially surrounding image elements, said circuitincluding providing means to provide at least two sets of surroundingimage elements similarity means to determine the difference betweenimage elements within each said set of surrounding image elements todetermine the similiarity between the elements therein respectively toselect a most similar set, dissimilarity means to discard sets of imageelements not sufficiently similar, and replacement means to fill in thevoid with at least one image element from the set of said at least twosets of surrounding image elements which contains the most similar imageelements.
 40. The circuit of claim 39 wherein said ranking means ranksthree or more sets of surrounding image elements by similarity.
 41. Thecircuit of claim 39 wherein said replication means utilizes multiplesets of surrounding image elements if said multiple sets are similar toeach other.
 42. The circuit of claim 41 wherein said replication meansonly utilizes multiple sets if said multiple sets each include at leastone adjacent and/or common image element.
 43. The circuit of claim 39wherein said replication means utilizes the average of the set of saidat least two sets of surrounding image elements which contains the mostsimilar image elements.
 44. The circuit of claim 39 wherein saidproviding means provides at least one set of image elements whichincludes elements on differing sides of the void.
 45. The circuit ofclaim 44 wherein the differing sides include the opposite sides of thevoid.
 46. The circuit of claim 39 wherein said providing means providesat least one set of image elements with adjacent image elements.
 47. Thecircuit of claim 46 wherein the adjacent elements include immediatelyneighboring image elements.
 48. The circuit of claim 39 wherein saidproviding means provides at least one set of image elements extendingacross adjacent fields.
 49. The circuit of claim 39 wherein saidproviding means provides at least one set of image elements extendingacross sequential fields.
 50. The circuit of claim 39 characterized bythe addition of determining means to determine if an image element isdefective and removal means responsive to said determining means toremove a defective image element to produce a void.
 51. The circuit ofclaim 39 wherein said providing means provides at least one set of imageelements based on time.
 52. The circuit of claim 39 wherein the imageelements are pixels.
 53. The circuit of claim 39 characterized by theaddition of removal means to remove an image element which is notdefective to produce a void.
 54. An improved signal processing circuitfor filling a void located at a position having at least partiallysurrounding image elements, said circuit including providing means toprovide at least two sets of surrounding image elements, similaritymeans to determine the difference betwen image elements within each saidset of surrounding image elements to determine the similarity betweenthe elements therein respectively, threshold dissimilarity means toblock the ranking of dissimilar sets of image elements, ranking meansfor ranking each set of surrounding image elements by similarity and,replication means responsive to said ranking means to fill in the voidwith at least one image element from the set of said at least two setsof surrounding image elements.
 55. The signal processing circuit ofclaim 54 characterized by the addition of a determining means, and saiddetermining means determining whether an image element is defective soas to create the void.
 56. The signal processing circuit of claim 54characterized by the addition of a removal means and said removal meansremoving image elements which may or may not be defective in order tocreate a void.